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Abstract 

This work explores a new robust approach for 
Semantic Parsing of unrestricted texts. Our ap- 
proach considers Semantic Parsing as a Consis- 
tent Labelling Problem (clp), allowing the in- 
tegration of several knowledge types (syntactic 
and semantic) obtained from different sources 
(linguistic and statistic). The current imple- 
mentation obtains 95% accuracy in model iden- 
tification and 72% in case-role filling. 



1 Introduction 

A central issue in Semantic Parsing is the produc- 
tion of a case-role analysis in which the seman- 
tic roles -such as agent or instrument- played by 



each entity are identified ( Brill fc Mooney 97 ). 
This is a crucial task in any application which 
involves some level of Natural Language Under- 
standing. 

This paper presents a new approach to Seman- 
tic Parsing (sp). The aim of this research is to de- 
velop a robust (able to work on unrestricted text) 
and flexible (portable and extensible) approach to 
Semantic Parsing. We will try to do so by formal- 
izing semantic parsing as an Consistent Labelling 
Problem (clp), specially focusing on the interac- 
tion between syntax and semantics, as well as on 
verbs, as the head sentence components. 



Mahesh ( Mahesh 93 ) proposes a classification of 
Natural Language Understanding models in: se- 
quential, integrated and interactive depending on 
the interaction between syntax and semantics. In 
Sequential models, each level receives the output 
of the previous one, and sends its output to the 
next. Thus, syntax is solved before any semantic 
analysis is carried out. On the other hand, in In- 
tegrated and Interactive models, syntactic and se- 
mantic processing are performed simultaneously 
and share a common knowledge representation. 

Another relevant issue related to syntax and 
semantics interaction is the required level of syn- 



of robustness presented by traditional full parsing 
approaches: close world assumption (full cover- 
age grammar and lexicon), local errors produced 
by global parsing considerations ( Grishman 95| ) 
and the selection of the best parse tree among 
a forest of possible candidates. Given that the 
verb is the central sentence component, there is 
no doubt that subcategorization information may 
not only improve parsing — e.g. taking into ac- 
count probabilistic subcategorization on a statis- 



tax analysis. Chunk parsing ( Abney 91 ) has been 
widely used in several fields (e.g. Information Ex- 
traction) as an alternative to deal with the lack 



tical parser (Carroll et al. 98), but also provide 
the basic information to assemble those chunks in 
larger structures. 

Following this brief introduction, sections [2], |3] 
and |] present, respectively, the basic ideas of our 
system and its architecture, as well as the way in 
which the different sources of knowledge are inte- 
grated. Section |5| describes the experiments car- 
ried out and reports the results obtained. Finally, 
section || presents some conclusions and outlines 
further research lines. 

2 pardon approach 

Our view of semantic parsing is based on compo- 
sitional semantics and lexicalized models (i.e. the 
meaning of a sentence is the result of combining 
the meaning of its words and the possible combi- 
nations are determined by their models). Bear- 
ing that in mind, pardon approach combines the 
Interactive Model and chunk parsing. Roughly 
speaking, pardon combines semantic objects as- 
sociated to chunks in order to build a case-role 
representation of the sentence. This combination 
is carried out using syntactic and semantic knowl- 
edge obtained from a linguistic approach (subcat- 
egorization frames) and complemented with a sta- 
tistical model of lexical attraction. 

For instance, starting from the chunks in the 
sentence "Este ano en el congreso del partido se 
hablo de las pensiones"Q shown in Figure p], we 

1 Literal Translation: This year in the meeting of the 
political party one talked about the pensions 
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Figure 1: Chunks for "Este ano en el congreso del partido se hablo de las pensiones" 



Este aho 



model:impersonal 
event: hablo 
se:se 

entity:de las pensiones 



modehNrfe 

head: en el congreso 

modif:del partido 



Figure 2: Case-role structures obtained for the sentence in Figure |l| 



will obtain the case-role representation shown in 
Figure Q by combining: 

• The initial semantic objects associated to 
those chunks 

• The impersonal model of the verb "hablar" 
(to talk) shown in Table [l] 

• The noun modifier model shown in Table |2| 

3 pardon architecture 

We propose a novel architecture where high level 
syntax and semantics decisions are fully inte- 
grated. There are two main steps in pardon: 
The first step is the Sentence Analyzer, which 
performs PoS-tagging, chunking and semantic an- 
notation. It also accesses the subcategorization 
(hand build) and lexical attraction (statistical) 
knowledge bases, to complete the sentence model 
with those kinds of knowledge. 

The verbal subcategorization information is ob- 
tained from LEXPIR ( Fernandez fc Marti 96| ; 
Fernandez et al. "9^ ; pVlorante et al. 98| ) developed 
inside the PirapidesQ project. Table [I] shows the 
basic and impersonal models for the verb "hablar" 
(to talk). In this work we used 61 verbs belonging 
to the LEXPIR trajectory class, which includes 
the equivalent of Levin's movement and commu- 
nication classes. 

The second step is Selection, which solves the 
Consistent Labelling Problem associated to that 



Pirapides is centered on the study of the English, 
Spanish a nd Catalan verbal p redicates, based on Levin 
(Levin 93) and Pustejovsky's (Pustejovsky 95) theories. 
LEXP1K models include information about the number of 
arguments, their syntactic realization, prepositions they 
can take, their semantic component and restrictions, their 
agreement with the verb and their optionality. 



sentence model to find out which is the most ap- 
propriate role for each chunk. 

4 pardon Formalization 

We formalize pardon approach by setting the 
case-role interpretation problem Consis- 
tent Labelling Problem (clp), where the different 
kinds of knowledge are applied as weighted con- 
straints. 

A CLP basically consists of finding the most 
consistent label assignment for a set of variables, 
given a set of constraints. Once the sentence and 
its knowledge is represented in terms of a CLP, 
a relaxation labelling algorithm is used to obtain 
the most consistent interpretation. See ( Padre 



98| ) for details on the use of these algorithm for 
nlp tasks. 

This formulation allows us to naturally inte- 
grate different kinds of knowledge coming from 
different sources (linguistic and statistic), which 
may be partial, partially incorrect or even incon- 
sistent. 

pardon represents the meaning of a sentence 
in terms of relationships between semantic ob- 
jects, using two variables for each semantic ob- 
ject: the model (object mo d e \) and role (object IO \ e ) 
variables. For instance, the semantic object as- 
sociated to a chunk headed by "hablar" (to talk) 
can use a basic model (someone talks about some- 
thing with someone: [habiar mo d e i = basic]) or 
an impersonal model (one talks about something 
[habiar mo d e i = impersonal]). 

The role variable represents the role that a se- 
mantic object plays inside the model of another 
semantic object. For instance, the semantic ob- 
ject "pensiones" (the pensions) can play the role 
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Table 1: Models for the verb "hablar' 



entity for both models of "hablar" (to talk) (e.g. 
\pensiones ro \ e = (entity, basic, hablar)]) 

To identify a role from a model label we need 
a triple (role, model, semantic object). For in- 
stance, the role starter of the basic model for 
"hablar", represented as (starter, basic, hablar). 

Since a CLP always assigns a label to all the 
variables, two null labels have to be added: the 
label none for the model variables (semantic ob- 
jects which does not have/use a model, usually 
leaf semantic objects with no sub-constituents) 
and the label TOP for the role variables (semantic 
objects not playing a role in the model of a higher 
constituent, usually the sentence head). Figure || 
shows the variables and labels associated to the 
semantic objects in Figure [I]. 



tent Labelling Problem, a set of constraints stat- 
ing valid/invalid assignations is required to find 
the solution, pardon uses three kinds of con- 
straints: The first group contains the constraints 
that encode the linguistic information obtained 
from verb subcategorization models. The second 
group are additional constraints added to force 
a tree-like structure for the solution. Finally, a 
third set of constraints encoding statistical infor- 
mation about word cooccurrences, was added in 
order to complement the subcategorization infor- 
mation available. 

Constraints are noted as follows: 

[A = x] ~ w [B = y] denotes a constraint stat- 
ing a compatibility degree w when variable A has 
label x and variable B has label y. The compati- 
bility degree w may be positive (stating compat- 
ibility) or negative (stating incompatibility). 



4.1 Subcategorization Constraints 

Two different kinds of subcategorization models 
have been used: one about verbal subcategoriza- 
tion and one about noun modifiers. 

For each chunk labelled as VP, all possible sub- 
categorization models for the verb heading the 
chunk are retrieved from LEXPIR. For PP and 
NP we use the simple nominal modifier model N^ e 
presented in table 

Due to the richness of natural language we can- 
not expect to find, in a real sentence sample, 
the exact prototypical subcategorization patterns 
that have been modelled in LEXPIR. Thus, a 
measure of the "goodness" of the possible model 
instantiation is defined in a similar way to the 
tree-edit based pattern matching used in ( [Atse- 
; Atserias et al. 00| ). 
In order to ensure the global applicability (min- 
imal disorder, agreement, maximum similarity be- 
tween the role and semantic object and maximal 
number of roles) and the consistence of the model 
(a unique instantiation per role and the instantia- 
tion of compulsory roles) the following constraints 
are automatically instantiated from the models: 

• Role Uniqueness: The same role can not 
be assigned to different chunks, e.g.: 

[pension vo \ e = (entity, basic, hablar)] 
[partidoroie = (entity, basic, hablar)] 

This constraint penalizes the current weight 
of the assignment [pension ro \ e = (entity, ba- 
sic, hablar)] according to the current weight 
of the assignment [partido vo \ e = (entity, ba- 
sic, hablar)]. Thus, the higher the weight for 
the latter assignment is, the faster the weight 
of the former will decrease. 



After formalizing Semantic Parsing as a Consis- r i as e ^ 9£ 
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Table 2: Model for noun modifiers 



• Model Support: A model assignment is 
compatible with its optional roles, e.g.: 

[hablar mo de\ = basic] 

[pension TO \ e = (entity, basic, hablar)] 

• Model Inconsistence: A model assign- 
ment is incompatible with the inexistence of 
any of its compulsory roles, e.g.: 

[habIar mo dei = impersonal] 

-i [se ro ie = (se, impersonal, hablar)] 

• Role Support: A role assignment is com- 
patible with the assignment of its model, e.g.: 

[pension ro ic = (entity, basic, hablar)} 

r^+sim(pension, (entity ,basic,hablar)) 

[habiar mo dei = basic] 

The weight for this constraint is defined as 
a function sim, which measures the similar- 
ity between two feature structures yielding 
a value normalized in [—1,1], inversely pro- 
portional to the number of relabelling opera- 
tions needed to transform one feature struc- 
ture into the other. Currently, only seman- 
tics, gender and number are considered. 

• Role Inconsistence: A role assignment is 
incompatible with the no existence of the as- 
signment of its own model, e.g.: 

[pension ro i e = (entity, basic, hablar)] 
-i [haWar mo dei = basic] 

Additionally, a special set of constraints has 
been introduced to deal with PP-attachment: 

• Local PP attachment: A prepositional 
phrase tends to be attached to its nearest 
head. The weight assigned to each con- 
straint will decrease along with the distance 
(in words) between the semantic objects in- 
volved, e.g.: 

[pension TO \ e = (entity, impersonal, hablar)] 

r^—distance(pension, hablar) J j 

4.2 Structural Constraints 

Some further constraints must be included to 
force the solution to have a tree-like structure. 
These constraints are not derived from the sub- 
categorization models. 



• TOP Uniqueness: Different assignments of 
the label TOP are incompatible, e.g.: 

[partido ro i = TOP] [haWar rok; = TOP]. 

• TOP Existence: There is at least a TOP. 
Notice that there is no right side on the con- 
straint as it is valid for any context, e.g.: 

[habiar rolc = TOP] « +1 [ ] 

• No Cycles: Two assignments forming a di- 
rect cycle are incompatible^], e.g.: 

[pension ro \ e = (modif, N^e, partido)} 
[partido r oie = (modif, Nd e , pension)] 

• NONE Support: The NONE model is com- 
patible with the inexistence of any role as- 
signment of the semantic object models, e.g.: 

[congreso modcl = NONE] « +1 

-i [pension ro ie = (modif, N^ e , congreso)] A 

-i \partido TO \ e = (modif, N^e, congreso)] 

If these constraints were not included, the 
NONE model would never be selected, since 
there would always be some other model with 
a very small non-zero support. 

4.3 Statistical Constraints 



In a similar way to ( Yuret 9q ) we define also a 
language model based on lexical attraction. In 
our case, we estimate the likelihood of a syntactic 
relation not between two words but between two 
semantic objects. 

Our hypothesis is that the relations between 
two semantic objects can be determined taking 
into account two special elements of their associ- 
ated chunks, the handle and the head. The han- 
dle of a chunk is usually the preposition which 
specifies the type of relation it has with another 
chunk, while the head of a chunk is supposed to 



capture the meaning of the chunk (Basili et al. 
p8|) . For instance, the chunk "de las pensiones" 
(about the pensions) has handle "de" (about) and 
head "pension" (pension). 

Since related words are expected to occur to- 
gether more likely than unrelated words, the 



3 In this first prototype of pardon indirect cycles are 
not taken into account 
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Figure 3: CLP associated to the objects in Figure |] 



lexical attraction (the likelihood of a syntac- 
tic relation) between two words can be es- 
timated/modeled through coocurrence. Co- 
occurence data can also indicate negative related- 
ness, where the probability of cooccurence is less 
than by chance. Thus, we will measure lexical 
attraction between two sematic objects with the 
coocurrence of both heads and the cooccurrence 
of the head and the handle (which gives an im- 
plicit direction of the dependence). 

Since the coocurrences were taken from the 
definitions of a Spanish dictionary, lemma co- 
ocurrences were used instead of word co- 
ocurrences in order to minimize the problems 
caused by unseen words ( Pagan et al. "99| ). 



head-head cooccurences were obtained out of 
40,591 different head-lemmas and 160 different 
handle-prepositions. The cooccurrences were 
used to compute Mutual Information for each 
lemma-preposition pair. 



M I (headi , handle j) = log 



P(headi n handle j) 
P(headi) x P(handlej 



In the case of lemma- lemma pairs, sparseness is 
much higher. Thus, an indirect measure was ap- 
plied, namely context vector cosine (also used in 
IR and WSD (Schtze, 1992)) in order to calculate 
the lexical attraction between heads: 



Hk a ki J2k a tj 



cos(headi, headj) 



where a pq is the cooccurrence frequency of lemma 
p and lemma q, and k ranges over all the lemmas 
cooccurring with any of both heads. 

Thus, for any two semantic objects the follow- 
ing constraints are added: 

• Ai-Hj constraint, which supports any assign- 
ment of a role from object j to objecti, e.g.: 

[partido ro \ e = (modif, N^, congreso)] 

r^M I (congreso,de) ^ j 

• Hi-Hj constraint, which supports any assign- 
ment of a role from objecti to object j, or 
viceversa, e.g.: 

[pension ro \ e = (entity, impersonal, hablar)] 

r^cos(hablar, pension) r 1 



Hi-Hj and A{- 



Hj constraints can be used to iden- 



175,333 head-handle coocurrences and 961,470 



tify adjuncts or relations for which we have no 
models. For instance, in the result obtained for 
the sentence shown in Figure |], the sematic object 
"en el congreso" (in the meeting) will be identified 
as depending on the verb "hablar" , even when its 
role can not be determined. 

5 Experiments 

170 real sentences were taken from a Spanish 
newspaper and were labelled by hand with the 
verbal models and the meaning components. The 
sentence average length is 8.1 words, ranging from 
3 to 23. Only one-verb sentences were selected, 
since our knowledge base does not include models 
for subordination or coordination. However, our 
approach to semantic parsing has been designed 
to manage multiple models simultaneously com- 
peting for their arguments. 



Each sentence in the corpus was tagged and 
parsed with a wide-coverage grammar of Span- 



ish (Castellon et al. 98) to obtain a chunk parse 



tree. Spanish Wordnet ( Atserias et al. 97 ) was 
used to semantically annotate the corpus with 
the 79 semantic labels defined in the prelimi- 
nary version of the Euro Wordnet Top Ontology 
flRodriguez et al. 98|) . 

In order to reduce the complexity of the re- 
laxation process, the possible role labels (which 
indicate the roles an object can play in any of 
the models retrieved) are filtered considering the 
unary constraints about POS and prepositions, 
while constraints about semantics and agreement 
are taken as a measure of how similar (sim) is the 
semantic object and the role. Models which can 
not match compulsory roles are not considered. 

For instance, the semantic object ano (year) 
in the example sentence will be allowed to match 
the role starter of the impersonal model of the 
verb hablar even though its semantics is not Hu- 
man, but the semantic object congreso will not 
be considered as a candidate to fill the entity role 
of hablar, since the preposition en in the semantic 
object does not match the model requirements for 
that role (preposition de, sobre). 

All these filters produce the candidate labels 
shown in Figure ||, and are the input to pardon 
Selection step. 

5.1 Results 

The results reported have been calculated using 
Message Understanding Conferences QMUC95| ) 
evaluation metrics applied to our particular case 
of verbal model identification and case-role filling. 

Model identification metrics evaluate how well 
our system identifies the right model for a seman- 
tic object. Our corpus has 2.7 models per verbal 
semantic object as average ambiguity. 

Since it is assumed that there is only one right 
model per chunk in each sentence, the answer can 
only be correct (COR) or incorrect (INC), thus, 
the used metrics are precision and recall. Table 
H shows the results obtained in the verbal model 
identification task: 95% precision and 91% recall. 
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Table 3: Verbal Model identification results 

Case-role filling consists in assigning each se- 
mantic object to the right role it plays in the mod- 



els for other semantic objects. In this case, the ca- 
suistics is more complex, since in addition to the 
correct /incorrect distinctions, other cases must 
be considered, such as the roles that are (cor- 
rectly/incorrectly) left unassigned (because they 
were optional, or there was no semantic object 
that fitted them, etc.). The MUC evaluation met- 
rics establish the following cases: 

• Correct (COR): Roles correctly assigned by 
the system. 

• Incorrect (INC): Roles incorrectly as- 
signed by the system. 

• Missing (MIS): Roles unassigned by the 
system when they should have been assigned. 

• Spurious (SPU): Roles assigned by the sys- 
tem when they should have been unassigned. 

These cases lead to the definition of the follow- 
ing measures, where Possible (pos) are the roles 
that should be assigned (cor+inc+mis) and Ac- 
tual (act) are the roles actually assigned by the 
system under evaluation (cor+inc+SPU): 



Undergeneration UND 
Overgeneration OVR = 
Substitution SUB = 100 x 
Error ERR = 100 x 
Precision PRE 
Recall REC 



= 100 x ¥m 
100 xf^ 

INC 
COR+INC 
INC+SPU+MIS 
COR+INC+SPU+MIS 



100 x^f 



lOOx^gf 



In addition, precision and recall may be com- 
bined in different F-measures (P&zR, 2P&R and 
P&2R). Table |] shows the results in the case-role 
filling for verbal arguments. 
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Table 4: Verbal case-role filling results 

To our knowledge there is neither a similar gen- 
eral approach nor case-role filling experiments to 
which our results can be compared. In any case, 
our preliminary results (72% PRE - 70% REC) 
are very encouraging. 



It is also remarkable that our system produces 
low values for UND, OVR and SUB measures, 
pointing that it properly uses the different kinds 
of knowledge, and that it does not take unin- 
formed or gratuitous decisions. 

Errors in the preprocessing steps caused most of 



i, INC) The miss- 
l MIS and SPU) 



miss-identified models (table 
ing and spurious roles (table 
were due either to the lack of semantic informa- 
tion or to the lack of a verbal model for adjuncts, 
which caused miss-identification of adjuncts as 
arguments, as in "(Juan) (esqm'a) (este fin) (de 
aho)"0, where the chunk "este fin de ano" (on 
New Year's Eve) is wrongly identified to fill the 
route role even though its semantics is Time. This 
is due to the lack of a selectional restriction that 
forces the route to be a Place, and to the lack of a 
model that identifies the chunk as time adjunct. 

6 Conclusions & Further Work 

This paper has presented a new approach to 
Semantic Parsing for non domain-specific texts 
based on the Interactive Model. The robustness 
and flexibility of pardon is achieved combining a 
chunk parsing approach with the framing of the 
semantic parsing problem in a clp. The flexibility 
of our approach enables the integration of differ- 
ent types of knowledge (linguistically motivated 
subcategorization models plus statistical informa- 
tion obtained from corpora). 

Currently, pardon obtains a 95% precision on 
model identification and 72% precision on role fill- 
ing. Although the experiments have been carried 
out on a limited corpus and lexicon, they have 
proven the feasibility of the method. 

Further work should approach a more realistic 
evaluation of the system, using a larger corpus 
with multiple-verb sentences. In this case, verbs 
will compete in a sentence for their arguments. 
We also plan to include more statistical knowl- 
ege (measures/language models) and to extend 
the coverage and expressiveness of the subcate- 
gorization models. Furthermore, the output of 
the current system could also be used as feedback 
to improve the existing verbal models. 

Exploration of linguistic and statistical mod- 
els for the identification/distinction of verbal ad- 
juncts should also be addressed, since it seems one 
of the main causes of the miss-identification of the 
verbal arguments. 



1 John goes skying on New Year's Eve 
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